Motion sensing display apparatuses

ABSTRACT

Motion-sensing display apparatuses supported near a user&#39;s eye including partially transparent screens at least partially disposed within the user&#39;s field of vision, image generators positioned to display an image on a first side of the screen, motion capture devices positioned near the screen and configured to capture a user gesture occurring beyond the screen in the user&#39;s field of vision, and processors in data communication with the image generator and the motion capture device, the processors configured to execute computer executable instructions in response to the user gesture. In some examples, motion-sensing display apparatuses include cameras. In some further examples, image generators display user interfaces on screens.

BACKGROUND

The present disclosure relates generally to motion-sensing displayapparatuses. In particular, motion-sensing display apparatuses supportednear a user's eye that execute computer executable instructions inresponse to user gestures are described.

Known motion-sensing display apparatuses are not entirely satisfactoryfor the range of applications in which they are employed. Some existingdisplay apparatuses are configured to interface with motion-sensingtechnologies, such as the Microsoft® Kinect® system. Many suchapparatuses, however, do not provide a display proximate a user's eye.As a result, these devices are often ill-suited for displaying imagesthat encompass large portions of a user's field of vision. Further,conventional motion sensing technology is generally large and notportable, limiting its use outside of a user's home or during physicalactivities.

Often, these displays include an opaque backing that prevents a userfrom seeing beyond the display. As a result, these devices are oftenunable augment or overlay scenes within the user's field of vision.Other devices may use a camera to capture a scene beyond the display andaugment this captured scene on a display. This often results inunnatural and unreliable displays, however.

Certain display apparatuses are mounted near a user's eye with a pieceof eyewear. These devices often produce a display that encompass a largeportion of a user's field of vision. These devices, however, ofteninclude an opaque backing, preventing the ability to augment naturalscenes and view user motions or gestures occurring beyond the screen.This, commonly in conjunction with the use of unwieldy accessories,leads to poor motion capture capabilities (if any exist at all).

Some other devices mounted on a piece of eyewear, such as Vuzix® brandeyewear devices, provide a display on a partially transparent screen.However, these devices often lack support for image capturing and motionsensing functionality. By extension, they often lack the ability toexecute computer instructions in response to motion or natural images,including the ability to manipulate the display.

Thus, there exists a need for motion-sensing display apparatuses thatimprove upon and advance the design of known motion-sensing devices anddisplay devices. Examples of new and useful motion-sensing displayapparatuses relevant to the needs existing in the field are discussedbelow.

SUMMARY

The present disclosure is directed to motion-sensing display apparatusessupported near a user's eye including partially transparent screens atleast partially disposed within the user's field of vision, imagegenerators positioned to display an image on a first side of the screen,motion capture devices positioned near the screen and configured tocapture a user gesture occurring beyond the screen in the user's fieldof vision, and processors in data communication with the image generatorand the motion capture device, the processors configured to executecomputer executable instructions in response to the user gesture. Insome examples, motion-sensing display apparatuses include cameras. Insome further examples, image generators display user interfaces onscreens.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a perspective view of a first example of a motion-sensingdisplay apparatus being worn by a user.

FIG. 2 is a perspective view of the motion-sensing display apparatusshown in FIG. 1.

FIG. 3 is a rear elevation view of the motion-sensing display apparatusshown in FIG. 1 showing a user performing a gesture and a screen, uponwhich an image generator is displaying a user interface.

FIG. 4 is a rear elevation view of the motion-sensing display apparatusshown in FIG. 1 showing the user performing a second gesture.

FIG. 5 is a rear elevation view of the motion-sensing display apparatusshown in FIG. 1 showing the user performing a third gesture.

FIG. 6 is a perspective view of a second example of a motion-sensingdisplay apparatus being worn by a user, the motion-sensing displayapparatus including a glove accessory.

DETAILED DESCRIPTION

The disclosed motion-sensing display apparatuses will become betterunderstood through review of the following detailed description inconjunction with the figures. The detailed description and figuresprovide merely examples of the various inventions described herein.Those skilled in the art will understand that the disclosed examples maybe varied, modified, and altered without departing from the scope of theinventions described herein. Many variations are contemplated fordifferent applications and design considerations; however, for the sakeof brevity, each and every contemplated variation is not individuallydescribed in the following detailed description.

Throughout the following detailed description, examples of variousmotion-sensing display apparatuses are provided. Related features in theexamples may be identical, similar, or dissimilar in different examples.For the sake of brevity, related features will not be redundantlyexplained in each example. Instead, the use of related feature nameswill cue the reader that the feature with a related feature name may besimilar to the related feature in an example explained previously.Features specific to a given example will be described in thatparticular example. The reader should understand that a given featureneed not be the same or similar to the specific portrayal of a relatedfeature in any given figure or example.

With reference to FIG. 1, motion-sensing display apparatus 100 isdesigned to display an image near a user 101's eye and execute computerexecutable instructions in response user gestures by incorporatingmotion and image capture features. As FIG. 2 illustrates, motion-sensingdisplay apparatus 100 includes an eyewear article 110, a processor 160,a lens 120, an image generator 130, a motion capture device 140, and amicrophone 150. Motion-sensing display apparatus 100 is configured tocapture user gestures, along with other objects within a captured scene,and execute computer processes in response to the captured objects. Thismay involve manipulating a display produced by image generator 130 onlens 120. Motion-sensing display apparatus 100 may additionally executecomputer processes in response to audio captured by microphone 150. Byincorporating these capturing technologies, motion-sensing displayapparatus 100 provides a new and useful way of displaying andcontrolling computer images and processes.

As FIG. 1 illustrates, eyewear article 110 supports motion-sensingdisplay apparatus 100 on user 101's head. Looking to FIG. 2, eyeweararticle 110 includes a first temple 112, a second temple 113, and a bar114 extending between the two temples. As FIG. 1 illustrates, firsttemple 112 and second temple 113 extend from eyewear article 110 to reston user 101's ears, providing support for motion-sensing displayapparatus 100.

As FIG. 2 illustrates, processor 160 is supported by bar 114. Processor160 is configured execute computer executable instructions, and may, byextension, control the electronic elements of motion-sensing displayapparatus 100.

Processor 160 may acquire data from other elements of motion-sensingdisplay apparatus 100 or from external sources and execute computerexecutable code in response to this data. For example, processor 160 isconfigured to acquire data from motion capture device 140 such as datathat corresponds to a user gesture. Processor 160 may additionally oralternatively acquire data from microphone 150. In some examples, theprocessor acquires data from a separate device, such as a portable musicplayer, a personal data assistant, a smart phone, a global positioningsystem, or the like.

Processor 160 is in data communication with image generator 130 and mayinstruct image generator 130 to generate and manipulate a displayprojected on lens 120. As FIG. 3 illustrates, processor 160 may, forexample, generate a user interface 170 on lens 120. Processor 160 mayacquire data from other elements of motion-sensing display apparatus 100and manipulate the user interface in response to this data.

As FIG. 1 illustrates, eyewear article 110 supports lens 120substantially near user 101's eyes. As FIG. 3 shows, lens 120 defines apartially transparent screen 122 at least partially disposed within user101's field of vision. Because screen 122 is partially transparent, auser is able to view objects beyond screen 122, which may include, forexample, gestures being performing and other objects within hisenvironment.

As FIG. 2 illustrates, image generator 130 is attached to first temple112 and is substantially aligned with screen 122. As FIG. 2 shows, asecond image generator, image generator 131, is attached to secondtemple 113 to project images on screen 122 distal image generator 130.This disclosure specifically contemplates positioning additional imagegenerators augment the expanse of generated images over a screen'ssurface area. Image generator 130 is configured to display images on afirst side of screen 122 facing user 101's eye in response toinstructions executed by processor 160. These images often may overlayand/or augment a naturally viewed scene within user 101's field ofvision. Image generator 130 may display both opaque and partiallytransparent images on screen 122.

FIG. 3 illustrates a representative set of images generated by imagegenerator 130; the generated images include a user interface 170 thatoverlays and augments a scene 102 within user 101's field of vision.User interface 170 may include one or more opaque or partiallytransparent screen elements that overlay a portion of scene 102 whileproviding user 101 a partially unobstructed view of scene 102. As FIG. 3illustrates, certain elements of user interface 170 allow user 101 toview elements behind displayed elements.

As FIG. 2 illustrates, motion capture device 140 is affixed to bar 114and is oriented away from user 101 when user 101 is wearing eyeweararticle 110 to capture images and motion occurring beyond screen 122. AsFIG. 2 shows, motion capture device 140 includes an image sensor 142including a camera, a light source 144, and a depth sensor 146. Motioncapture device 140 captures scene 102, which comprises everythingviewable by motion capture device 140. FIG. 3 illustrates an example ofsuch a scene.

As FIG. 2 shows, image sensor 142 is supported by bar 114 and orientedaway from user 101 as user 101 wears eyewear article 110. Image sensor142 is configured to capture an image from a scene and communicate datacorresponding to the captured image to processor 160. Image sensor 142is not limited to capturing light within the visual spectrum. Imagesensor 142 may, for example, capture infrared or ultraviolet lightreflected on a scene from an artificial light source.

As FIG. 2 shows, light source 144 is supported by bar 114 and issubstantially aligned with image sensor 142. Light source 144 isconfigured to project light onto a scene. This light is reflected fromthe scene and may be captured by motion capture device 140. This mayallow motion capture device 140 to more accurately capture images andmotion present within the scene. Light source 144 is capable ofprojecting visual light as well as light outside the visual spectrum,which may improve capturing accuracy without altering the scene'snatural appearance.

As FIG. 2 illustrates, depth sensor 146 is supported by bar 114 and ishorizontally spaced from image sensor 142. Depth sensor 146 isconfigured to capture images and motion from a scene at a differentangle than image generator 130. The data captured from this secondviewpoint allows processor 160 to compare the data communicated by depthsensor 146 to the data communicated by image sensor 142 to better detectobjects' depth in the captured scene.

As FIGS. 3-5 illustrate, motion capture device 140 may capture imagesand motion that correspond to a user gesture, such as a gesture 191, agesture 192, a gesture 193, and/or a gesture 194. Upon capturing agesture, motion capture device 140 communicates data corresponding tothe gesture to processor 160, which executes computer executableinstructions in response to received data.

For example, FIG. 3 illustrates a first hand 105 of user 101 performinggesture 191 as image generator 130 generates user interface 170including a radio button 180 adjacent to a caption reading “OPENBROWSER.” User 101 performs gesture 191 by extending two fingers beyondscreen 122 to align with a selected screen element, such as radio button180, in the user's field of vision. As user 101 extends his fingers,motion capture device 140 captures this motion and communicates datacorresponding to this gesture to processor 160. Processor 160 thenexecutes computer executable instructions in response to this data. Inthis example, processor 160 selects radio button 180 in response togesture 191.

FIG. 3 illustrates a second hand 103 performing gesture 192 in concertwith gesture 191. As gesture 192 shows, motion-sensing displayapparatuses may capture multiple gestures being operated in concert. AsFIG. 3 shows, gesture 192 is similar to gesture 191, wherein hand 103extends two fingers beyond screen 122 and aligned with a button 181reading “OPEN WORD PROCESSOR” in user interface 170. As user 101 extendsfingers of his second hand, motion capture device 140 communicates datarelating to this gesture to processor 160, and processor 160 executescomputer executable instructions to launch a word browsing applicationin response.

FIG. 4 illustrates gesture 193, which is captured and processed similarto gesture 191. User 101 performs gesture 193 by extending a singlefinger to align with a screen element, window 182, at a first position195 within user interface 170 and moving the extended finger to alignwith a second position 196 within user interface 170. In response,processor 160 moves window 182 from its original position proximatefirst position 195 to a second position proximate second position 196.

FIG. 4 illustrates gesture 194, which is also captured and processedsimilar to gesture 191. User 101 performs gesture 193 by extending allof his fingers to align with user interface 170 and swiping the fullhand horizontally across screen 122. In response, processor 160 changesthe display on screen 122 from a first display window 185 to a seconddisplay window 186, wherein each display window includes a collection ofscreen elements. In this example, first display window 185 includes abutton for “OPEN BROWSER,” which is displayed prior to swiping the fullhand and removed after Likewise, second display window 186 includes abutton for “OPEN WORD PROCESSOR” in this example, which appears afterswiping the hand. This is designed to provide similar functionality toscreen switching available on many mobile phone platforms.

As gesture 192 illustrates, motion-sensing display apparatus 100 iscapable of distinguishing a user from the remainder of a captured scene.For example, motion-sensing display apparatus 100 may distinguishbetween different users, such as when a person other than the userperforms a second user gesture in the view of motion capture device 140.FIG. 1 displays such an example, wherein both user 101 and a second user199 are performing gestures. Motion capture device 140 is configured tocapture user 101's gestures as well as second user gesture 198. Thisdisclosure additionally or alternatively contemplates gestures thatincorporate multiple movements, which may be performed by multiple partsof a user's body. For example, gestures may incorporate actions by auser's hands and/or feet. Processor 160 configured to execute computerexecutable instructions in response these captured images and motions,which may include second user gesture 198.

Processor 160 may additionally store captured data to learn a user'sfeatures. In certain examples, users may verify processor 160's userdifferentiating functionality to train processor 160 to betterdifferentiate users.

As FIG. 3 illustrates, motion-sensing display apparatus 100 is notspecifically restricted to capturing user gestures. Motion-sensingdisplay apparatus 100 may capture any element of a captured scene andcommunicate data related to that element to processor 160. Processor 160may then execute computer executable instructions in response to thecaptured element. For example, FIG. 3 illustrates motion capture device140 capturing a scene including a tree 104. As FIG. 3 shows, processor160 executes computer executable code to display a dialog 106 inresponse to capturing the tree. This disclosure specificallycontemplates gesture based interaction with dialogs displayed inresponse to captured elements in a scene, such as dialog 106.

As FIG. 2 illustrates, microphone 150 is supported by bar 114.Microphone 150 defines an opening substantially aligned with motioncapture device 140 that allows sound to reach microphone 150. Microphone150 captures audio data as motion capture device 140 operates.Microphone 150 communicates captured data to processor 160, which mayexecute computer executable instructions in response. Processor 160 mayact in response to the audio data either alone or in concert with datacaptured by motion capture device 140. Processor 160 may, for example,respond to a user's vocal commands or respond to noises present inmotion-sensing display apparatus 100's environment.

With reference to FIG. 6, a second example of a motion-sensing displayapparatus, motion-sensing display apparatus 200, will now be described.Motion-sensing display apparatus 200 includes many similar or identicalfeatures to motion-sensing display apparatus 100. Thus, for the sake ofbrevity, each feature of motion-sensing display apparatus 200 will notbe redundantly explained. Rather, key distinctions betweenmotion-sensing display apparatus 200 and motion-sensing displayapparatus 100 will be described in detail and the reader shouldreference the discussion above for features substantially similarbetween the two motion-sensing display apparatuses.

As FIG. 6 shows, motion-sensing display apparatus 200 provides amotion-sensing display substantially similar to motion-sensing displayapparatus 100. Motion-sensing display apparatus 200 includes a processor260 and a motion capture device 240, each substantially similar to thecorresponding elements of motion-sensing display apparatus 200.Motion-sensing display apparatus 200, however, additionally includes amotion capture accessory 299.

Motion capture accessory 299 is configured to augment motion capturedevice 240 in capturing the gesture. Motion capture accessory 299defines a glove in data communication with processor 260. As FIG. 6shows, this disclosure additionally or alternatively contemplates theuse of more than one accessory; motion-sensing display apparatus 200includes, for example, a second accessory 298 defining a glove on theopposite hand of motion capture accessory 299. Both accessories mayinclude movement sensors, such as accelerometers or gyroscopes, tocapture data relating to user movement and communicate this data toprocessor 260. This allows processor 260 to respond to user movementmore precisely.

As FIG. 6 shows, motion capture accessory 299 includes a transmitter 297configured to communicate wirelessly with a receiver 296 proximate toand in data communication with processor 260. This allows motion captureaccessory 299 to communicate any captured data to processor 260.

Accessories may additionally include elements that allow motion capturedevice 240 to track its position more accurately. Such elements may bebodies attached to the exterior of the accessories. Accessories mayadditionally or alternatively be constructed of a material selected forincreased compatibility with motion capture device 240. Such accessoriesmay assist motion capture device 240 in capturing the movement,location, and shape of the relevant parts of the user or of theaccessory without requiring a data connection with processor 160.

The disclosure above encompasses multiple distinct inventions withindependent utility. While each of these inventions has been disclosedin a particular form, the specific embodiments disclosed and illustratedabove are not to be considered in a limiting sense as numerousvariations are possible. The subject matter of the inventions includesall novel and non-obvious combinations and subcombinations of thevarious elements, features, functions and/or properties disclosed aboveand inherent to those skilled in the art pertaining to such inventions.Where the disclosure or subsequently filed claims recite “a” element, “afirst” element, or any such equivalent term, the disclosure or claimsshould be understood to incorporate one or more such elements, neitherrequiring nor excluding two or more such elements.

Applicant(s) reserves the right to submit claims directed tocombinations and subcombinations of the disclosed inventions that arebelieved to be novel and non-obvious. Inventions embodied in othercombinations and subcombinations of features, functions, elements and/orproperties may be claimed through amendment of those claims orpresentation of new claims in the present application or in a relatedapplication. Such amended or new claims, whether they are directed tothe same invention or a different invention and whether they aredifferent, broader, narrower or equal in scope to the original claims,are to be considered within the subject matter of the inventionsdescribed herein.

The invention claimed is:
 1. A motion-sensing display apparatussupported near a user's eye, the motion-sensing display apparatuscomprising: a partially transparent screen at least partially disposedwithin the user's field of vision; an image generator positioned todisplay an image on a first side of the screen; a user interfacedisplayed by the image generator on the screen comprised of at least oneelement which overlays so as to obscure a portion of the scene naturallyviewed through the screen; a motion capture device positioned near thescreen and configured to capture user gestures occurring beyond thescreen in the user's field of vision wherein a first user gestureinteracts with at least one of the overlaid user interface elements, anda second user gesture interacts with at least one of the overlaid userinterface elements; and a processor in data communication with the imagegenerator and the motion capture device, the processor configured toexecute computer executable instructions in response to the first usergesture in concert with the second user gesture.
 2. The motion-sensingdisplay apparatus of claim 1, wherein the processor is configured toexecute computer executable instructions to manipulate the userinterface in response to user gestures.
 3. The motion-sensing displayapparatus of claim 2, wherein the processor is configured to executecomputer executable instructions to adjust the position of a selectedelement of the user interface in response to the user moving a fingerfrom a first point to a second point spaced from the second point,wherein the second point is aligned with the user interface within theuser's field of vision.
 4. The motion-sensing display apparatus of claim1, wherein the motion capture device includes a motion capture accessoryconfigured to augment capturing the gesture.
 5. The motion-sensingdisplay apparatus of claim 4, wherein the accessory includes a glove. 6.The motion-sensing display apparatus of claim 4, wherein the accessoryis in data communication with the processor and is configured tocommunicate with the processor wirelessly.
 7. The motion-sensing displayapparatus of claim 1, wherein the processor is configured to executecomputer executable instructions for distinguishing the user from theremainder of a captured scene.
 8. The motion-sensing display apparatusof claim 7, wherein the processor executes computer executableinstructions in response to a portion of the remainder of the capturedscene.
 9. The motion-sensing display apparatus of claim 1, wherein theprocessor is configured to distinguish between gestures made by the userand gestures made by a person other than the user, and execute computerexecutable instructions in response to the gestures made by the personother than the user.
 10. The motion-sensing display apparatus of claim1, further comprising an eyewear article configured to support thescreen and the motion capture device.
 11. The motion-sensing displayapparatus of claim 1, wherein the motion capture device includes animage sensor.
 12. The motion-sensing display apparatus of claim 11,further comprising a light source producing illumination substantiallyaligned with the image sensor, wherein the light source improves theaccuracy of the motion capture device without altering the naturallyviewed scene's appearance.
 13. The motion-sensing display apparatus ofclaim 12, wherein the light source produces light outside the visualspectrum.
 14. The motion-sensing display apparatus of claim 12, whereinthe image sensor includes a depth sensor configured to detect theillumination produced by the light source.
 15. The motion-sensingdisplay apparatus of claim 1, further comprising a microphone in datacommunication with the processor, wherein the processor is configured toexecute computer executable instructions in response to audio capturedby the microphone.
 16. The motion-sensing display apparatus of claim 1,wherein the second user gesture is performed by a different hand thanthe hand performing the first user gesture.
 17. The motion-sensingdisplay apparatus of claim 1, wherein the at least one of the overlaiduser interface elements the second user gesture interacts with isdifferent from the at least one of the overlaid user interface elementsthe first user gesture interacts with.
 18. A motion-sensing displayapparatus for interpreting a user's motion to control softwareapplications, comprising: an eyewear article including a substantiallytransparent lens; an image generator in a position to display an imageon the lens; a user interface displayed by the image generator on thescreen comprised of a plurality of elements which overlay so as toobscure a portion of the scene naturally viewed through the screen; amotion capture device comprising a camera positioned proximate thescreen and configured to capture user gestures made in front of thelens; and a processor in data communication with the image generator andthe motion capture device; wherein a first user gesture performed by afirst hand interacts with at least one of the plurality of elements ofthe overlaid user interface elements, a second user gesture performed bya second hand and in concert with the first user gesture interacts witha second of the plurality of elements of the overlaid user interfaceelements, and the processor is configured to execute a first set ofcomputer executable instructions in response to the first user gesture,and a second set of computer executable instructions in response to thesecond user gesture, and the motion capture device can be configured todistinguish between gestures made by the user and gestures made by aperson other than the user.
 19. A motion-sensing display apparatus forinterpreting a user's motion to control software applications,comprising: a partially transparent screen at least partially disposedwithin the user's field of vision; an image generator in a position todisplay a user interface on the screen; a user interface displayed bythe image generator on the screen comprised of at least one elementwhich overlays so as to obscure a portion of the scene naturally viewedthrough the screen; a motion capture device mounted near the screen andconfigured to capture a user gesture occurring beyond the screen in theuser's field of vision and interacting with at least one of the overlaiduser interface elements; and a processor in data communication with theimage generator and the motion capture device, the processor configuredto manipulate the user interface in response to the user gesture, andconfigured to execute computer executable instructions to select ascreen element generated by the image generator of the user interface inresponse to the user extending two fingers in parallel and substantiallyaligning the parallel extended fingers with the selected element in theuser's field of vision.